language models

Perplexity

2026-10-22 06:50:24

Perplexity and NVIDIA introduce JOLT, a post-training method that cuts compute use by 6.5x

Perplexity and NVIDIA researchers have introduced JOLT, a new post-training method for language models that assigns a single model two roles at once: teacher and student. In this setup, the teacher has access to privileged information while the student does not. The main change is that the teacher is also trained, which the researchers say keeps its feedback usable during the process. According to MarkTechPost, JOLT matched the accuracy of GRPO on the MATH benchmark while using 6.5x fewer training tokens. The team also presented JOLT+, a version that adds rewards based on the student’s own outcomes. In the reported tests, JOLT+ posted the best average score across all settings. On AppWorld, JOLT+ reached a pass rate of 72.09%, compared with 62.20% for GRPO, a gap of 9.9 percentage points. The method has been validated on multiple models, including Qwen3 and DeepSeek-V2-Lite.

300

Asian AI

2026-10-20 04:04:02

Tech in Asia: Asian AI language models face weak paid demand

Techub, citing Tech in Asia, reported that AI labs in Asia had hoped local-language capabilities would give them an edge against major technology companies. The data cited in the report points the other way on the business side: users have shown little willingness to pay for those products. As a result, many of these projects are being kept alive through philanthropy and corporate social responsibility, or CSR, support rather than clear commercial demand. The report centers on a gap between technical positioning and revenue reality, with local language specialization failing to translate into meaningful paid adoption. That has left a number of AI initiatives in the region leaning on outside support to continue operating.

550

Sakana AI

2026-10-12 18:03:00

Sakana AI says self-improving model neared a 70% win rate after two refinement rounds

Sakana AI, an AI startup, said its self-improving model uses the MASS method to let a single language model improve itself by solving tasks, judging outputs, and refining its workflow. According to the report cited by Techub News, the model’s win rate in benchmark testing rose to nearly 70% versus the base model after two rounds of improvement. The research also flagged a limitation. In some cases, the model showed blind spots around certain errors. If the same mistake appears across all task roles handled by the model, the training process may treat that error as progress and reinforce it instead of correcting it. The original source for the item was The Decoder.

600

AMD

2026-07-25 03:18:25

AMD unveils fully open-source Instella-MoE, a 16B-parameter model aimed at top-tier open models

AMD on July 25 introduced Instella-MoE, a fully open-source mixture-of-experts language model with 16 billion total parameters and 2.8 billion active parameters per token. The company said the model was trained from scratch entirely on its own AMD Instinct MI300X and MI325X GPUs, using the ROCm software stack, and incorporates architectural designs including Gated Multi-head Latent Attention, or Gated MLA, and FarSkip-Collective to improve training and inference efficiency. According to AMD, the Instella-MoE-16B-A3B base model posted an average score of 76.7 in performance tests, placing it among the leading fully open-source models and ahead of models such as SmolLM3-3B and OLMo-3-7B. AMD also said the model can compete with larger systems while activating only 2.8 billion parameters. Instella-MoE supports 64K-token long-context processing and has gone through a full training pipeline that includes pretraining, mid-training, long-context extension, supervised fine-tuning, direct preference optimization, and reinforcement learning. AMD said it is releasing the full model weights, training configurations, data mix, intermediate checkpoints, and inference code.

4350

Google

2026-07-23 11:30:15

Google Open-Sources DiffusionGemma, Claiming 4x Faster Text Generation at the Cost of Quality

Google DeepMind has released DiffusionGemma under Apache 2.0, saying it can generate text about four times faster than comparable autoregressive Gemma models, while public benchmark scores remain below standard Gemma 4.

750